Draft
Conversation
Next Steps to MergeImportant checks have failed. As of today they are not blocking this PR, but in near future they may.Addressing the following failures is highly recommended:
Comment generated by summarize-checks workflow run. |
API Change CheckAPIView identified API level changes in this PR and created the following API reviews
|
slister1001
commented
Jan 6, 2026
| source: OpenAI.EvalJsonlFileContentSource; | ||
|
|
||
| /** The ID of the taxonomy to use for seed generation. */ | ||
| taxonomy_id: string; |
Member
Author
There was a problem hiding this comment.
remove taxonomy id, will be passed into source
…r taxonomies - Add taxonomy_version optional field to TaxonomyBasedEvaluatorDefinition for version pinning (supports reproducible evaluations and audit trails) - Add taxonomyId query parameter to ListEvaluatorVersionsParameters for filtering evaluators by taxonomy ID
Add cleaner REST endpoints for discovering resources related to a taxonomy:
- GET /evaluationtaxonomies/{name}/datasets - List datasets generated from taxonomy
- GET /evaluationtaxonomies/{name}/runs - List evaluation runs that used taxonomy
- GET /evaluationtaxonomies/{name}/evaluators - List evaluators referencing taxonomy
This provides a more RESTful alternative to the existing query parameter approach.
…n flow evaluators/models.tsp: - Add 'behavior' to EvaluatorDefinitionType union - Add BehaviorBasedEvaluatorDefinition model with: - behavior_name: string - behavior_definition: string (10-2000 chars) - taxonomy_id: string (read-only, set by service) - taxonomy_version: string (read-only, set by service) evaluation_taxonomies/models.tsp: - Add 'behavior' to EvaluationTaxonomyInputType union - Add BehaviorTaxonomyInput model - Add subtopics field to TaxonomySubCategory This enables the simplified "evaluator-first" flow where users can create an evaluator with a behavior definition inline and the taxonomy is automatically created behind the scenes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Choose a PR Template
Switch to "Preview" on this description then select one of the choices below.
Click here to open a PR for a Data Plane API.
Click here to open a PR for a Control Plane (ARM) API.
Click here to open a PR for only SDK configuration.